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ABSTRACT . • ' - . 

A general aodel along with four illustrations is 
presented for the consideration of budgetary constraints in the 
setting of passing scores in instructional prbgrams involving 
remedial action for poor test performers. . Budgetary constraints , 
normally put ^n uppur limit on -any choice o^ passing score. Given 
relevant information,, this limit may be determined.^ Alternately, ways 
"to assess the budgetary conseguences .associated with a given passing 
^corrye are provided. Such information would be useful in any final 
decision regardjlng tte passing score; (Author) 
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ABSTRACT . > 

A gttnttral modal along with fou;r Illustration* la prasantad for 
tha cooaldaration df budgatary constraints In tha tatting of passlnjt 
scoras In InstrucipKMil prograns Invqlvlng ramadlal {ictlon for poor 
tast pai^foroars; Budgatary constraints normally put an uppar limit 
on* any cholca passing scora. Glvan ralavant Infbrmatloni^ this 
limit may ha datftrmlnad. AlttmAt«Iy» ways to asaaaa tha budgatary 
conaaquai^cas aaaoclatad vlth a glvan -patainy acora ara provldad. 
Such Information would ba usaful In any final daclslon i^agardlng tha 
passing scora. 
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INTRODUCTION 



In many Instructional programs, such «^s Individually Prascrlbad 
Instruction (Glaiiar» 1968) or othars of a similar natura ^Atkinson, 
1968; Flanagan, 1967), tasting Is conductad at tha and of avary 
Instruculonal unit to provlda <^aadback to tha s.tudant and/or CAachar 
in ordi'r (hat -«pproprlata actlvsn can.ba takan. If a studant^s 



scora Is high-, lt;f^y ba raasonabla tp grant tl:iat studant mastary 
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of. the current unit and to alloy him to proceed to a subsequent 

unir. On the other hand/a low score may Indicate that the studeftt 

mlgWt bettef it;^|||om some retfl^dial action. This is also the case for 

certif ijcation tfMting such as high school graduatldn oc for minimum 

competency testing aa legislated in several states. Funds are 

« 

usually allocated for rtmediation'^ for students whose scores are too 
Ibw to warrarit mastery of the x^orapete^cles under conslderAtlon. 

The statistical issues relating to granting or denying mattery 
status have been approached by several writers, ' indludin^Huynh 
\l976. 1977, 1978). Most proposed schemes jire by And large quotf- 
, free, i.e'. , the maatery/nonmastery decision process consideted by* . 
Che writers does not take into account the budgetary consequences ^ 
associated wli|ih the denial of mastery status. If funds provided' . 
tor remediation are limited , then a constraint will have to be * 
Imposed on the number^ of .stud ent;s declared as failures (noniHastfers) . 
/ The purpose of this paper is to demonstrate how budgetary 
res trie tiotisv -may be taken Into account in the process o£ seating 
paa^ling (mastery) • scores or perf ormance/standards. • Alter na'tely, 
the^ presentation provides ways to' assess the budgetary cons^tquences" 
associated with an arbitrary passing score. Section 2 describes 
the overall framework. Illustratiofls based on the beta-binomial 
^nd normai--normal test scord models ^wiUL be provided in subsequent 
sectilons. ^ \ \ * ' 

• f 2 . OVERALL > FRAMEWORK 

, It is now assumed 'thajt the true ability*'of a population of 
subjects may> be described by a random variable e which ranges in. 
the sampiLe space YJ. For the beta-binomial model, 6. is th^ propor- 
tion of itfims that a subject answers correctly \in an item pool aYid* 
Q ±B the interval f rom ^O^to 1. ' For the: normal test score model, 9 
is the traditional true 'score. (Lord & Nc^vfck, 1968) and Q h the 
entire real line. Let the prpbablli^y density function (pdf) .of '9 
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Let X be the scrore obtained ft'otn. the /idmlnist;i:at Ion of an n- 
Item teat and let f(x) and f(x!e) denote Ita marginal and condi- 
tional probability density functions with Vespect to 9» 

I It shall be assumed that all subJecCs with <BBt scores smaller 
than a passing (mastery) score c will be denied mastery f<9r the 
ytnstructioijal objectives covered by the test and that these subjects 
4^will be provided with appropriate temedlal learning activities, 
the remediation is assumed to be so devSied that^lts conclyslon 
wilf coinc4^de wTth the ma|it>ery status which was previously denied 
the student. The cost of remediation will be assumed to be a non-- 
increasing function of ,0 ahd will be denpted as 5(0)* Thus, 
remediation will coat less for more' able students than it will for 
less able ones. * ^ 

*• ^ ponsider now a subject with trtie ability 0. The probability ' 
that this person will be declared in need of re^^Bdiat:lon is given 
as the* sum i;f(x|e) or* the integral / f(xj9)dx, with x < c. For the 
purposes of this section, the sunmiatio^ notation will be used. It 
follows that the (conditional) expected remediation cost for tftis 
subject is ^ » ' 

^ , Z f (x|9)5(e) . , • • . - • . ' 

x<c ^ ^ 

H^nce the (unconditional or marginal) expected refnediatiori cost for 

' a subject drawn randomly from the populatidn is ^ 

y(c) • Z f (x|9)6(9)p(9>d9. . , , (1) 

x<c * ' 

This function i^ nondecreaslng with rjBspect to its argument c. .Its 

' lowest limit is zero (when all subjedts are granted mastery statu?) * 

and its maximum v^lue, y • ''/o <5(9)p(9)d0, is reached when 

max w ^ 

remediation is provided to all ^ubjetts regardless ^f their test: 
' scores. / ' ^ 

Let us suppose, furthermore, that testing is to be conducted 
for a total of m subjects and the total cost Of possible remediation 
cannot exceed the value B* If the parsing score c is selected^ then, 
the total f^xpected remediation cost will be mY(c). Hence anyjcholce 
for c must satisfy the budgetary constraint my(c) £ B. If ^. B, 
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any cutoff 8Cor« will b« jacc«pttbl«. However » if B < y^^f then 
the passing score c tnust be less than or equal to c^, y^here is ^ 
the highest score satisfying the Inequality 

y(Cj^) < B/m. (2) 

For discrete test scares, such as those of the binomial error model. 

Inequation (2) may be solved by computing the values of yCo) on« by 

ope, starting** with c as the smallest test score, and stopping when 

the value c, is reached. For continuous test data, numerical' pro- 

1 - ■ • , ^ ♦ • 

cedures for solving the nonlinear equa-tion y(cj^) " might be 

« • 
needed. ^ . * . . . ' . 

3. nrHE BETA-BI NOhqAL MODEL WITH CONSTANT COSTS " 

— t-^ — i i 1 — ^- — . " ( 

ConsicJer now the beta-bifibmlal model as defined by the follow- 
ing pdf's: 



f(x|9) - (")e''(i-e)""'''. X - 0,lv...,n 

ii4^ii|li-i, 0 < e < 1. 

B(a, 0^ ' 

The two toarameters a and $ may be estimated from sample data via 
one o*f ieveral estimation tecliniques such as the moment procedure 
ot the maacimum likelihood procedure. Let % and s be the sample 
fpt score mean and standard deviatlQAr"'' In addition, let a^^ be 
the KR21 reliability coefficient as defined by * 



and 



n 



1 > ?.. < ; "''x) 



.2 
ns 



(3) 



(In the case of a negative- Ojj^, simply replace the value computed 
from" Equation CJ) by any positive, reliability estimate.) -The moment 
estimates for a. and 6 are given as ^ 

, . a - C-1 + 1/021^'^ ' ■ ^ ^ .^^^ 

V 

-and . ■ % ' 

e - -a + n/ij^ - n. ^ ^ (5) 

We will now focus on the simple case where a single true pass- 
Ing score (or criterion level^e^, separating true 'masters from 



BUDGETARY CONSIDERATION 



trutt nonmfflt«rSy yima b«en spaclfl^d. Let the rexn«dlati6n cost be 
conatJtnt and equal to for a true nonmaster and lo^x true 
maater. Thua the coat funirtlon la of the form 



y^ If e < e 



The nonlncreaalng nature of 5(6) la satisfied whenever y > y- • 

The expected remediation cpat per atudent^s shown Equa-y 

^ tlon (1) la nbw given aa ^ 

• c-l r 1 

^^^^ - ^ V h ^9 ^ 

x-o • ^ o 

9 



■•de 



Y(c) 



n+0-x-l 



o 'o 



B(a,6) J„ V 



de 



Yj,B(a+x,n+3-x) * 



^ 9 



It may b« noted that the marginal beta-blnomlal pdf of x Is given as 

f(x) - (")B(ot+x,n+6-x)/B(a,8) , (6) 

and that the Incomplete beta function I(a-hc',i*f6-x;9 ) Is def lived as 



9 



o 



I(cffx,n+e-x;9^) - T 9 



(1-9) 



n+6-x-l 



d«^(a+x,irt-6-x) 



It follows" that , 

c-l " > 
y(c) - E f(x)(Y3^ + (YQ-Yi)l(oi-Hc,n+6-x;9^)} . 

The values of f(x)" may be computed via the following indu^ctiye 
formulae: ' , ^ ^ ' . 



(7) 



f(0) 



IT 

i-1 



' n+ctf 8-1 



and 



f(x+l^ tW (x+1) (n+6-x-l^ 



0 9 1 9 • • • I n— 1 • 



(8) 



(9) 



6 
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Th« following reourrnnce fonmila, on the other hand, will quicken 
Che evaluation of 'the Incomplete beta functions: 

Q^^d Q\n+6-x-l 
I(.H^l.nH-e-x-l;e/. - ■ (^)Vcrbc,nH.8-x) ^•K'^.'^-B-x-. 9„) ■ 

• (10)' 

Finally, aa In Section 2, let B be tjhe maximum funds allocated 
for possible remediation Involving a S^oup of m subjects. Then the- 
paasing score cannot exceed the highest integet Cj^ at which 
y(c.) B/m. ^ - - ^ 

Numerical Example 1 * ; ^ 

A maximum sum of B - $4000 has been »llo<iated for ^remediation 
in an instructional program with m - 100 studenW. Thus B/m • $40* 
For the program under study, assume that 6^ « •60 and the remedia- 
tion costs are y $i50 for each student with true ab"llit;y 0 .60 

and Y- «• $50 for student^ with 9 > .60. Nowsupposii a 5-rl tern .test 

JL • ^ * 

is administered andi the test scores yield the estimates a 3 aad 
6-2. At the passing scores c » 1^ 2, 3, 4, anid^5, th« expected 
remediat^n coats yCc) are $7.02. $19,06, $31.83, $41.25, and , 
$47.1^, respectively^. Since, yic^) <_ $40, it follows that Cj^ • 3. 
The- budget constraint imposes an upper limit of 3 on the passing 
score. If 3 is used, the expected cost of remediation amounts to 
$3183. If the next higher passing' score,* 4, were used, the expected 
remediation cost would be $4125, over the maximulm budgeted sum ^f 
$4000. ■ . * - 

4. THR BETA-BINOMIAL MODEL Wim..lINEAR COSTS 

Let us 8up))ose now th«»t the cost; function may be written as - • 
HQ) -'(Y^-YiXl-e) + O (11) 

in which Y< < Y • Thus the cost is a linear function offO-, It is^ 
1 o . ^ 

eaqal to y when 9 p 0 and y-i when 9 "1. ^ ' <• 

.Under the beta-binomial model as described in the first para- 
'graph of Section 3, the expected cost per student is givea as 
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C-1 



X"0 



^VV Q <i-Q) 



" '"^ (Y^-Yj^^B(otfoc,n+'0-x+l) + Y]^B(otfx,n+e-x) 



^ („) 



B(«, B)' 

x-o 

JBy noting that ■ - • ' 

B(ctfx,n+e-x+l) - °|f7o - B(a+x,n+6-x) 

It may b« deduced that . ' ' 

c-1 (y -Yi ) (iH-6-x) 

x-o » 

c-1 . a (n+e-x) + Yi (ot+x) , 

, - ^ f« - — • - ■ 

X"0 • • 

As in the previous, section, the values of f (x) may be, computed 



inductively via Equations (8) ftnd (9). 

Numerical Example 2 \^ 

Consider the basic data of the first numerical example, namely 
B/m - $40, Yj5 " $150, Y-j^ " $50, a - 3, B " 2, and n. - 5 items. At 
the passing scores of 1, 2, 3, A, and 5, tife ejfpected remediation' • 
costs y(<i) are $5:71, $18.81, $37.86, $59.29, and $78.33. Hence 
the passing score cannot exoee'd 3, whet« the maximum value of the 
expected cost of remediation would amount to $3786# Had a score 
of 4 been selecteci, the expected cost would*" have amounted' to as 
much as $59*29. ' ' « ^ w ' ' 

To close thiaf' section, it should belmentioned that simple 
expressions for y(c) such as 'the one of EqOation (12) may be worked 
out for all cost functions 6(9) which can be represented as inte- 
C gral polynomials of 9. ^" *^ ' ' ^ 

- 5. THE BXVARIATE NORMAL MODEL WITH CONSTANT COSTS 

Now consider* the case where the^true score 9 and the observed 

* ' 'A ^* 

score x*are jointly distributed according to a bivariate normal 



8 O "^^^^^^ 



distribution. Without any lo8S of generality, it may be assumed 
that X is in its standardized form with zero mean and unit variance. 
Let p be the reliability of the^^test for the normal population of 
subjects under consideration. The true score 0 has. a mean of zero, 
a standard deviation of and a correlation of /q witK^he test 
score X- The joint pdf of x and 9 is * 

2 



f(x,9) ■ exp 

2it/p(1-p) , 



(x^ - 2x6 + ^) 



2(l-p) p;j^ 

As in Section 3, it will be assumed that the cost function 



(13) 



6(6) is constant, taking the values of for 6 ,< 6^ and the valu^ 

of Y, for 6 > 6 . It follows from' Equation (1) that at any passing 
1 o 

score c, th^ remediation cost for a subject drawn randomly from 

the population is expect;;pd to be ' . " 

.y(c) - / ° f(x,e)4edx + T /q f(x.e)dedx . ' - 

—00 --00 —00 Q 

e 

- Y.Pr(x < c) + (y -Y.) f / f (x,e)d6dx., (14) 

-00 ^00 / . ^ V . 

* The maximum passing score c- satisfies the equation^ y(Ci ) • B/m. 
This value of c^ exists -as long as B < y^^^^ where 

' Y - Y Pr(6 ^ 6;^) + YiPr(6 > 6^) , 
. 'max ^o ^o '1 — o 

Solutions may be found via numerical procedures' such as the 
Newton Iterative solution "for nonlinear equations. To apply this 
technique, it may be noted that the derivative of y{c) with respect 
to c is , 

y'Cc) - Yifj,(c) + (y^-y^ f(c,e)de ' ■> 

wh^e fjj(*) denotes the pdf of x (the unit normal variable). In 
" Other words,* 

2t 

It may* also be noted that 9 ^ ' ' 

* V i • ■ • - 
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0 



/ ° f(c.9)de -I f^(c-) . 



6 ~pc 
o 



Vp-p 



\>h«r« F„(.4 is th<j (cumulative) distribution fuhction the unit 
N . 

nohtOtl .variable, 
tn summary, 



y'(c) f^(c) 



^1 ^ V^P^N 



9 -pc 



(15) 



Both Y(c)-and y'(c> may be evaluated via computar programs such as 
thoaa described in the IMSL (1977)* They may also be obtained by ^ 
use of appropriate tables for the univariate and bivartat^ normal 
distri'butions. ' ' 

Numer^g ^l Examjple 3 

Let the parameters defiiling the problem be p « .64, 9^ * 1> 

V « $150, • $50, and B/m - $40. Numerical procedure yieldi the 
o 1 

maximum standardized passing sdore c * --•475* If the tesit scores. 

i * ' 



4iave a mean of ^50 and a aitandatd deviation of 20, then the passing 
score cannot exceed 40.5.. 

6. the bivxriate normal model 
With normal-ogive cost 



/ 



Now consider the case where the cost function 6(6) la of the 



form 



6(6) 



1 - F 



N 



9-€L 
o 



(i6) 



whtfre, fa before, Fjj(') represents tHe distribution function of the 
unit; normal variable. In. the context of decision" theory, expres- 
sions slml^lar to those of Equatiqn (16) haye been, proposed as 
utility function's (e.g., Lindley, 1976, and Novick and Lindley, 
1978). As in the case of the 'beta-binomial model with linear costs, 

Y and represent the remediation oosts associated with the least 
o 1 • 

able (9 i« -«>) and the most able (9 - +<») subjects. On the other . 

♦ 

hand, tfte parameter 9^r^® the locatibn at ^/hich the cost is y-. 
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(y +Yi )/2^nd^/a indicates the extent to which 6(8) decreases at 
o 1 . 

this location. 

, The expected remedlatioh cost y{c) may now be written as- 
c -f«> 

. Y(c) « / / f U,,e)5(e)d6dx 

^ —00 —00 



- Y Pr(x < c) - (y -Yi) J <(>(x)f (x)dx 

O — 0 1 N 



(17) 



where 



+« 



4>(x) - / f(9|x)Fj^ 



e-e 



de 



The conditional pdf f(0|x) is given as 



f(8jx) - 



/2ttp(1-p) 
It follows that 



exp 



. (Q-px) 



[ 2p(l-p)J 



A +C0 



^ 2Tra/p(l-p) -« 
It should b^noted that the expression 



[ .2p(l~p)J i 



(t^e )^ 

' ^ o 



2a 



dt}de 



2ira/p(l~p) 



- exp 



^ 2 
on2 (t-9 ) 

(e-pk) _ o 



2p(l-^ 



2a 



acts as th« joint pdf of two indep^dent normal random variables 9 
* . 2 ■ / 

and t with means px and and with variances p(l-p) iand a . 

Now let us introduce the new random variable u «• 6 - t for 

2 2 

which the mean is px - 6 and the variance is p - p + a . Si^ice 

• o • 
the condition t < 6 is equivalent to u > 0,' it follows- that <J>(x) 

may be expressed sltaply as ^ . 



eu' 

where g. (e,u) is the bivariate normal pdf of 6 and u. Jience 
9u 

(J»(x) • Pr(u >^ 0) - 1 - Pr(u < 0) - " »' 



or 
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11 



♦(x) - 1.- F 



N 



9 -px 
o 



J 272 
/p-p +a 



with this n«w expression for (t>(x) , the expected remedlatl^ 
cost as defined in E<matlon (17) may he written as 



Y(c) - V^Pr(x < c) + (Yq-Y^) / 



/p-o +a • 



fj,(x)dx. 



(19) 



The Integral found In Equation (19) may be vrrltten aa 
c h(x) 

Z(c) - / J / f^(w)f^(x)dwdx. 



where h(x) ^(-PX^V)^!P^ and fj^(.-) Is again the pdf (5f a unit 

normal variable. ' Let: r ' ' . v / 

V « w - h(x) » w + (px-,9 )//p-p +a • V ^ ^, / 

Then x and v follow a joint blvarlate notjnal pdf, (xW,'^with ^ 

* . * XV 

means, va^lanaes, and.-correlatlon given*, respectively , as * 



/ 2 2 



CT^ - (p+a^)/(p-p^+a^) , 



(20) 



and 



Pj^ " p//p+a . j 
Hence the integral Z(c) takes a simpler for* given as 

Z<=) - i / g' (x.v)dvdx, 

and the expected remediation cost y(c) may be written as 

c o 

^Y(c) - Y3^Pr(x < c) + (Yq-Yj^) J / g^(x,v)-dvdx. 



f 



(21)* 



^ 00 —00 



The numerical values of yicY may be computed^ via tables or 
computer programs dealing with the univariate and bivariate normal 
distributions. ^ * . . 



12 



miYNH 



Numerical procedures such as the Newton Iteration procfess may 
bejised to splve the equation y(c) " B/pi. The derivative t>f y(V) 
with 'respect^ to c, from Eqitatlon (19), la fomid to be . ^ 



Y'(c) -^^(c) 



Yi + (Y^-Yi)Fj^ 



a -pc 

o . 



■/ 2^ 2 



(22) 



It may be noted that by taking - 0, Equations (19) and (22) 
of this sectien will reduce to Equations (14) and (15) of Section 5. 
This* is expected since the normal-ogive cost futjiction 5(9) fs . 
defined in<(16y will degenerate into the constant cost function of 
Section 5 when a tends to zero. Finally^ the maximum expected 
remediation cost (per random subject) may be deduced from Equation 
(21) by letting c - is 



m 



6 



4 



p+a 



(23) 
* 



Numerical Example 4 ' • 

Let the parameters of the problem be p - .64, 9^ » 1, a - 2, 
Y " $150, Y-, ■ $50, and B/m - $40. The Newton iteration procedure 

O J X 

for solving the equation y(c.j^) ■ B/m yields the solution c^ » --.362. 
If the test scores have a mean bf 50 and a standard deviation of 20, 
thet)' the test passi^ng score cannot excee4 42.76. 

7. SOME CONCLUDING REMARKS 

In this paper a general model along with four separate illus- 
trations is provided for the consideration of budgetary constraints 
in the setting of passing scores in instructional programs involv- 
ing remediation for subjects with poor test performance. The 
illu8trationSv_are not meant to be exhaustive. Budgetary constraints 
normally Impose a limit on the number of students allowed to take 
remedial learning activities 'and, hence, restrict the range in 
which a choice for the passing score is to be made. The paper also 
provides ways to assesd the budgetary requirement associated with 
each passing score. This information .would be a factpr in deci- 
sions regarding passing scores and budgets for remediation. 
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